Convergence rate of moments in stochastic approximation with simultaneous perturbation gradient approximation and resetting
نویسندگان
چکیده
منابع مشابه
Convergence rate of moments in stochastic approximation with simultaneous perturbation gradient approximation and resetting
The sequence of recursive estimators for function minimization generated by Spall’s simultaneous perturbation stochastic approximation (SPSA) method, presented in [25], combined with a suitable restarting mechanism is considered. It is proved that this sequence converges under certain conditions with rate O(n ) for some >0, the best value being = 2=3, where the rate is measured by the Lq-norm o...
متن کاملConvergence of simultaneous perturbation stochastic approximation for nondifferentiable optimization
In this paper, we consider Simultaneous Perturbation Stochastic Approximation (SPSA) for function minimization. The standard assumption for convergence is that the function be three times differentiable, although weaker assumptions have been used for special cases. However, all work that we are aware of at least requires differentiability. In this paper, we relax the differentiability requireme...
متن کاملSimultaneous perturbation stochastic approximation of nonsmooth functions
A simultaneous perturbation stochastic approximation (SPSA) method has been developed in this paper, using the operators of perturbation with the Lipschitz density function. This model enables us to use the approximation of the objective function by twice differentiable functions and to present their gradients by volume integrals. The calculus of the stochastic gradient by means of this present...
متن کاملCorrection to "Optimal random perturbations for stochastic approximation using a simultaneous perturbation gradient approximation"
The simultaneous perturbation stochastic approximation (SPSA) algorithm has recently attracted considerable attention for optimization problems where it is di cult or impossible to obtain a direct gradient of the objective (say, loss) function. The approach is based on a highly e cient simultaneous perturbation approximation to the gradient based on loss function measurements. SPSA is based on ...
متن کاملBudget-Dependent Convergence Rate of Stochastic Approximation
Convergence rate results are derived for a stochastic optimization problem where a performance measure is minimized with respect to a vector parameter θ. Assuming that a gradient estimator is available and that both the bias and the variance of the estimator are (known) functions of the budget devoted to its computation, the gradient estimator is employed in conjunction with a stochastic approx...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Transactions on Automatic Control
سال: 1999
ISSN: 0018-9286
DOI: 10.1109/9.763206